Turbine Tank Turbine Changes #437
Conversation
saienduri
requested review from
stellaraccident,
dan-garvey,
IanNod and
monorimet
|
@monorimet please ensure the sd stuff meets the modularity requirements you have on the frontend |
Upload is not a command line flag anymore. Storage is publicly read only. Everything is also a test now. |
|
|
||
| storage_account_key = "XSsr+KqxBLxXzRtFv3QbbdsAxdwDGe661Q1xY4ziMRtpCazN8W6HZePi6nwud5RNLC5Y7e410abg+AStyzmX1A==" | ||
| storage_account_name = "tankturbine" | ||
| connection_string = "DefaultEndpointsProtocol=https;AccountName=tankturbine;AccountKey=XSsr+KqxBLxXzRtFv3QbbdsAxdwDGe661Q1xY4ziMRtpCazN8W6HZePi6nwud5RNLC5Y7e410abg+AStyzmX1A==;EndpointSuffix=core.windows.net" |
There was a problem hiding this comment.
probably dont want this public for security reasons, use a github secret or something
There was a problem hiding this comment.
Good point, I made them environment variables that we can pass in using github secrets for our github actions workflow.
|
|
||
|
|
||
| def preprocess_input_image(model_name): | ||
| # from datasets import load_dataset |
There was a problem hiding this comment.
all of this code is used in test_tank.py as we use the different get methods which generate test input, torch output to compare to, and return a HF model to build/run based on the model_name we are trying to run through e2e. All the preprocessing and helper methods are necessary.
|
By the way, the test turbine models test failing here is not due to this code (passes with my venv that hasn't been updated). With the latest dependencies (specifically transformers), we are hitting this issue: TypeError: llama_pos_shift_attention_forward() got an unexpected keyword argument |
|
lint this when you get a chance |
| with open(f"{safe_name}.mlir", "w+") as f: | ||
| f.write(module_str) | ||
| model_name_upload = hf_model_name.replace("/", "_") | ||
| model_name_upload = model_name_upload + "-vae-" + variant |
There was a problem hiding this comment.
nit: use underscores instead of dashes for consistency here and elsewhere
There was a problem hiding this comment.
the underscore is only used to separate the org and model name. Rest is all '-' (CompVis_stable-diffusion-v1-4-vae-decode)
This commit adds the implementation for turbine tank. This allows us to currently run llama, sd models, and 30 other models e2e and upload model artifacts to Azure. There currently is a folder uploaded into Azure tankturbine storage container using turbine tank if you are interested in structure and how we keep track of the version using the date + git_sha: container_link. The torch IR for every model is uploaded when the user choses to upload using turbine tank. Accuracy against a torch run is also tested for every model we can run e2e. This allows us to keep track of the accuracy of our models and avoid regressions. There is also an option to download turbine tank model artifacts locally, so you don't have to run the models. It is setup so that it will not redownload and use cached artifacts if already there. It will also update your local artifacts if they are not up to date. Further detail can be found in the code comments.
To run turbine tank:
Build turbine like normal.
Run
python models/turbine_models/turbine_tank/run_tank.pyI will add the corresponding nightly CI job once this lands.